In videoconferencing applications, a video stream (e.g., webcam video) is typically sent from a presenter/sender to multiple participants/receivers. Depending on various bandwidth requirements and the capabilities of the various receiver devices, different receivers may request different resolutions or different bit rates of the presenter's video. In order to address this issue (and to prevent the sender from having to encode different individual video streams for each receiver) the sender typically encodes a single stream which is dispatched by a video server and subsequently split into several individual streams having frame rates suitable for the various receivers. For example, for a video conferencing stream from a sender, a first receiver may request VGA video at 30 frames per second and a second receiver may request VGA video at 15 frames per second. In this scenario, the sender would encode the video at 30 frames per second to meet the requirements of the first receiver for the video server which would in turn “scale” the video (using a rate controller) so that it drops every other frame to meet the requirements of the second receiver. However, there are several drawbacks associated with the aforementioned scaling approach. One drawback associated with current rate controllers is that they must independently control a quantization parameter for multiple layers utilized in the scaling of video. However, current rate controllers are unable to accomplish this without introducing flicker which results in less than optimal video quality for the receivers. It is with respect to these considerations and others that the various embodiments of the present invention have been made.